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ABSTRACT 

Gaining  greater  understanding  of  the  requirements  or  needs  of  a  commander  or  Operation  Analysis  Team 
for  modeling  human  behavior,  and  the  effectiveness  of  modelling  and  simulation  for  training  for  the 
Afghan  mission,  require  a  modelling  process  to  make  the  structure  of  the  decision  domain  explicit.  As  an 
initial  knowledge  elicitation  step  in  this  process,  we  have  conducted  a  pilot  study  involving  two  field 
studies  to  create  an  initial  model  of  how  a  team  in  a  vehicle  patrol  operates  within  a  training  scenario 
implemented  using  a  virtual  training  system  at  the  Swedish  Armed  Forces  International  Centre 
( SWEDINT ')  at  the  Swedish  Life  Guard  Regiment  in  Stockholm.  The  scenarios  designed  by  the  instructors 
at  IntUtbE,  the  pre-deployment  training  for  peace  support  operations  unit  within  the  Life  Guard  Regiment, 
are  used  to  train  and  prepare  teams  for  the  real  environment.  Most  importantly,  the  simulation-based 
training  process  creates  an  opportunity  to  train  a  team  for  specific  possible  events  that  can  occur  in  the 
field,  and  how  to  act  if  those  events  arise. 

The  method  used  for  modelling  cognitive  processes  involved  in  this  simulated  (and  real)  operational 
environment  began  with  the  Belief,  Desire  and  Intention  (BDI)  model  of  human  decision-making.  BDI  has 
typically  been  used  to  model  individual  motivational  factors  and  cognitive  decision  processes.  Since  patrol 
activities  involve  groups,  the  BDI  model  has  been  extended  with  several  layers  to  bring  the  team  aspect 
into  focus. 

The  first  field  study  within  the  pilot  study  involved  observation  of  a  simulated  patrol  and  recording  of  the 
patrol  session  using  voice  recording,  video,  notes  taken  by  an  observer,  and  the  screen  history  of  the 
simulation  together  with  captured  eye  gaze  data  upon  the  screen  using  an  eyetracking  system.  The 
eyetracker  provides  tracking  of  objects  of  visual  attention  of  the  patrol  leader,  and  dialogue  between  the 
patrol  leader  and  driver  can  be  analyzed  in  relation  to  these  objects.  The  second  field  study  omitted 
eyetracking  for  practical  reasons. 

The  simulation  used  in  the  field  studies  has  been  developed  by  SWEDINT  using  Virtual  Battle  Space 
(VBS),  a  simulation  engine  developed  by  Bohemia  Interactive.  With  this  system,  each  team  member  has  a 
health  bar.  One  proposal  following  from  this  study  is  that  it  could  be  effective  in  improving  and 
empowering  the  team  aspect  of  training  to  provide  a  multidimensional  Team  radar'  to  show  the  changing 
status  of  a  team  according  to  several  measures  of  state  and  performance  beyond  a  simple  health  bar.  The 
team  radar  would  consist  of  a  graphical  circle  with  different  radius  indicators  for  factors  such  as  trust, 
physical  state  and  courage,  as  well  as  the  arsenal  and  equipment  available  for  the  team  roles  and  the 
team  as  whole.  Comparing  a  target  team  radar  state  with  the  evolving  state  during  training  would 
potentially  provide  an  effective  monitor  of  training  progress.  Such  a  display  would  also  be  interesting  to 
investigate  for  operational  applications,  e.g.  using  augmented  reality  technology. 
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1.0  INTRODUCTION 

With  ISAF  troops  coming  from  approximately  50  countries,  modelling  and  simulation  (M&S)  is  a 
powerful  tool  for  training  in  operational  effectiveness  and  cultural  awareness  by  providing  safe  synthetic 
experiences  of  situations  prior  to  deployment  in  theatres  of  operation.  M&S  can  mediate  human  support 
for  small  groups  engaged  in  deep  knowledge  building.  It  can  empower  groups  to  construct  forms  of  group 
cognition  that  exceed  what  the  group  members  could  achieve  as  individuals.  However,  the  effectiveness  of 
M&S  systems  needs  to  be  validated  and  systems  improved  to  take  advantage  of  evolving  technologies  and 
understanding  of  human  cognitive  functions.  This  paper  presents  a  pilot  study  including  two  field  studies 
of  current  M&S  training  for  ISAF  staff  and  also  of  command  and/or  an  Operation  Analysis  Team  involved 
in  modelling  human  behavior  in  training  scenarios.  The  aim  is  modelling  individual  and  group  decision 
processes  within  the  training  scenario  as  a  knowledge  elicitation  process.  Such  cognitive  decision  models 
can  represent  both  intended  and  actual  learning  outcomes,  e.g.  for  purposes  of  evaluating  the  effectiveness 
of  M&S  based  training,  and  can  also  provide  foundations  for  operational  systems,  e.g.  for  human  support, 
group  and  organizational  coordination,  or  as  a  foundation  for  decision-making  by  autonomous  agents 
within  operational  systems. 

In  going  from  empirical  investigation  of  M&S  training  situations  to  derived  cognitive  models,  the  starting 
point  has  been  the  AI  modelling  approach  called  the  Belief  Desire  and  Intention  (BDI)  model  invented  by 
Bratman  (1999/87).  The  initial  BDI  model  has  been  enhanced  with  a  planning  layer  by  Georgeff  and 
Ingrand  (1989).  Wooldridge  (2000)  uses  the  BDI  model  because  it  combines  three  important  elements:  1) 
it  is  founded  upon  a  well-known  and  highly  respected  theory  of  rational  action  in  humans,  2)  it  has  been 
implemented  and  successfully  used  in  a  number  of  complex  fielded  applications,  and  3)  the  theory  has 
been  rigorously  formalized  in  a  family  of  BDI  logics.  However,  critiques  of  the  original  model  have 
included  that  it  is  lacking  in  both  learning  and  social  (here,  primarily  team)  aspects  of  cognition 
(Wooldridge,  2000).  The  study  reported  here  is  a  pilot  study  since  it  has  been  necessary  to  investigate  the 
strengths  of  the  BDI  model  and  to  specify  potential  extensions  or  modifications  required  for  effectively 
modelling  decisions  within  ISAF  processes.  The  result  is  a  series  of  cognitive  modelling  approaches 
derived  from  BDI,  presented  in  the  paper.  Some  potential  uses  of  these  models  are  also  proposed.  Actual 
implementation  of  the  models  could  be  based  upon  different  cognitive  architectural  models  such  as  SOAR 
(State,  Operator  And  Result,  Laird  et  al,  1987),  ACT-R  (Adaptive  Control  of  Thought — Rational, 
Anderson,  2007),  or  programming  languages  such  as  PROLOG,  LISP,  C++  etc..  However,  the 
implementation  of  the  models  is  beyond  the  scope  of  the  research  reported  in  this  paper. 


2.0  METHOD 

Initial  studies  of  gaze  behavior  in  tactical  simulations  preceding  the  study  reported  here  were  concerned 
with  details  of  visual  cognition  in  relation  to  visual  design  during  combat  (see  Sennersten,  2007).  The  two 
field  studies  documented  here  zoom  out  from  this  earlier  work  to  understand  how  a  group  of  people 
having  different  roles  make  decisions  from  within  the  framework  they  are  operating  in.  Gaze  logging  is 
used  again  within  the  first  pilot  study  and  dialog  analysis  is  added  as  a  basis  for  analysing  social 
interaction  and  team  cognition  with  respect  to  roles  and  rules  in  typical  training  scenarios. 

The  operational  situation  chosen  for  the  field  studies  is  ground  patrol  operations  in  an  Afghan  village/town 
conducted  with  the  use  of  one  or  more  four  wheel  drive  military  ground  vehicles.  This  scenario  is  part  of  a 
larger  training  process  for  people  joining  the  International  Security  Assistance  Force  (ISAF)  in 
Afghanistan.  The  first  field  study  included  two  instructors  operating  the  simulation  as  trainees,  a  person  in 
charge  of  the  simulation  platform  and  one  technician.  This  study  was  mostly  a  practical  exercise  to  record 
gaze  with  a  traditional  set-up  with  an  eyetracker  without  interconnecting  it  with  the  simulation  engine.  The 
simulation  scenarios  were  created  in  VBS2™,  see  Fig.l.  This  study  also  contained  sound  and  pulse 
recording  to  find  out  how  the  communication  between  the  participants  developed  (pulse  data  is  not 
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considered  in  detail  in  this  paper).  This  first  study  also  had  the  aim  of  understanding  via  communication 
with  instructors  the  detailed  purpose  and  design  of  the  scenario  created  by  SWEDINT  for  IntUtbE 
simulation-based  training  of  officers  going  to  join  the  International  Security  Assistance  Force  (ISAF)  in 
Afghanistan;  the  primary  training  would  occur  some  weeks  after  the  field  studies  reported  here. 

For  the  first  field  study,  several  conversations  were  held  over  several  months  between  IntUtbE,  the 
primary  author  of  this  paper  and  also  SWEDINT  staff  in  charge  of  their  simulation  platform.  A  couple  of 
visits  were  arranged  to  meet  staff  and  go  through  what  kind  of  questions  were  of  interest  and  why  in 
relation  to  this  research.  Mail  correspondence  and  phone  calls  were  used  to  sort  out  the  details  and  provide 
answers  in  between  physical  meetings.  The  field  study  examined  training  for  two  instructors  who  were 
new  to  the  simulation  system,  prior  to  its  use  for  training  their  own  trainees. 

The  second  field  study  observed  20  officers  from  Sweden  and  Norway  participating  in  simulation-based 
training  using  the  same  simulation  scenarios  as  in  the  first  field  study.  This  second  field  study  was  not 
visually  recorded  but  concentrated  more  upon  social  interactions  via  observation  and  sound  recording.  The 
greatest  value  from  this  simulation-based  training  session  was  obtained  from  the  answers  to  questions  the 
participating  officers  gave  retrospectively  (the  analysis  of  these  questions  can  be  found  in  Sennersten, 
2010). 

In  terms  of  simulation  design,  the  instructors  design  the  scenarios  together  with  the  simulation  platform 
staff  and  want  to  shape  emergency  situations  so  the  officers  can  prepare  themselves  mentally  for  what  can 
happen  when  they  are  in  a  real  operational  situation.  The  aim  is  also  to  practice  with  team  members 
together  where  many  have  not  met  before,  to  practice  navigation  and  radio  communication  and  when  to 
contact  the  Tactical  Operational  Centre  (TOC).  Evaluation  happens  routinely  in  the  form  of  an  After 
Action  Review  (AAR),  retrospectively  after  a  simulation-based  training  session.  An  emergency  situation 
may  occur  due  to  language  issues,  so  language  and  cultural  differences  with  the  context  are  also  important 
to  practice  in  a  forgiving  synthetic  environment. 

The  simulation  platform  staff  design  scenarios  beforehand  and  also  creates  simulation  details  on  the  fly 
during  simulation  sessions.  Following  a  training  session,  instructors  summarize  performance  and  discuss 
certain  hardships  and  successes  during  the  simulation  session(s)  with  trainees,  what  kind  of  scenarios  the 
teams  met,  the  roles  and  interpersonal  interactions  involved.  Video-captured  material  from  VBS2™  can 
also  provide  additional  information.  In  relation  to  BDI  and  plans  it  is  good  to  know  what  initial  plan,  in 
the  form  of  a  flap  sheet  and  certain  pre-specified  routines  the  staff  are  asked  to  perform,  so  the  plan  can  be 
the  basis  for  debriefing  and  for  determining  what  was  learned  during  the  simulation  session.  SWEDINT 
technical  staff  and  the  person  in  charge  of  Simulation,  Modeling  and  the  Practical  Platform  see  these 
training  environments  as  providing  two  major  outcomes1:  i)  to  support  command  exercises  and  to  provide 
stimuli  for  international  Brigade  Staff  courses  and  ii)  training  for  future  international  peacekeeping 
missions.  A  foremost  aim  is  to  train  and  exercise  cultural  awareness,  language  (i.e.  English),  and 
collaboration  with  an  interpreter  and  team  during  decision  making.  Visualization  helps  to  understand  and 
to  comprehend  dialog  when  a  different  language  is  being  used  in  contact  with  local  inhabitants.  Hence 
visualization  helps  to  unify  participants  when  describing  what  is  happening  and  also  in  aiding  participants 
in  more  quickly  shaping  a  collective  mental  comprehension  together. 


2.1  Experiment 

Two  experiments  were  conducted  with  the  same  scenarios  designed  by  SWEDINT  based  upon  an 
operational  idea  specified  by  IntUtbE  and  implemented  by  SWEDINT  "s  technical  staff  at  the  Simulation, 
Modeling  and  Practical  Platform. 


1  Data  from  field  study. 
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Figure  1 :  Screenshots  of  UK  MoD  in  VBS2  vl.22. 


2.1.1  Tools  and  Equipment 

The  tools  and  equipment  set-up  (see  Fig. 2)  available  for  the  field  studies  reported  in  this  paper  include: 


SWEDI  NT  Setup 


VBS 

Computer/Client  - ►  UserScreen 


BTH  Setup 


VBS 

Computer/Client 


Eyetracker 

Screen 


Figure  2:  Connection  scheme  for  VBS  server/client  +  screen 
and  VBS  server/client  +  eyetracking  screen. 


Tobii  Eyetracker  T60 

Simulation/Game  Engine  Virtual  Battle  Space  2,  VBS2™running  on  3  PC's 

Sound  recording  freeware 

Simulation  scenarios  in  VBS2™  editor 

POLAR  pulse  watch 

Keyboards 

Extra  interaction  devices:  Steering  wheel  and  pedals 
Analogue  (paper)  map 
Walkie  Talkies 
Microphones 

Control  room  with  instructors  role  playing  and  making  simulation  design  on  the  fly 
Excel,  Word 
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2.1.2  First  Field  Study 

In  the  first  field  study,  training  staff  were  selected  who  were  able  to  evaluate  the  simulation  and  its  value 
as  a  training  environment.  To  look  at  decision  making,  the  Patrol  Leader  (PL)  is  the  natural  choice  for 
eyetracking  because  s/he  is  responsible  for  the  group  and  its  decisions.  Communication  tasks  within  the 
vehicle,  outside  and  via  radio  make  this  role  the  key  to  the  effective  performance  of  the  whole  group.  Of 
course,  every  other  role  has  its  own  responsibilities  and  challenges,  but  will  be  carried  out  in  accordance 
with  a  main  decision  by  the  PL.  The  PL  shall  not  only  secure  her  or  his  own  survival,  but  also  that  of  the 
team,  and  achieve  the  patrol  objectives  and  treat  people  with  respect  and  optimism  so  survival  is 
reinforced  for  all. 

Two  people  are  in  the  patrol  vehicle,  one  driver  and  one  PL.  For  the  training  session,  each  role  is  played 
by  separate  participants,  who  sit  next  to  each  other  in  the  simulation  room,  as  shown  in  Fig. 3.  In  the 
control  room  there  are  two  instructors  and  some  other  people  voice  acting  in  dialog  when  digital 
characters/  avatars  are  approached  in  the  simulation  (see  Fig  4). 


Figure  3:  Officers  preparing  for  simulation-based  training,  instructor  in  the  middle. 


2.1.3  Experimental  Task 

The  experimental  task  was  to  gather  gaze  data,  audio  recording,  video,  and  make  observations  in  support 
of  subsequent  cognitive  modeling  of  decision  processes  involved  in  the  operational  patrol  task. 


Figure  4:  The  team  in  this  study  was  limited  to  the  patrol 
leader  (who  is  also  the  navigator)  and  the  driver. 
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In  a  computer  simulation  with  stimuli  on  a  screen,  the  team  members  gaze  upon  the  screen  in  order  to 
navigate  and  meet  challenges  in  the  simulated  environment.  Fig.  5  shows  the  distribution  of  PL  gaze 
behavior  over  a  simulation  session. 


Figure  5:  Heat  map  of  visual  attention  from  within  the  car,  for  the  patrol  leader. 


Figure  6:  A  predesigned  route  here  contains  8  waypoints. 


Figure  7:  PL's  first  person  view  of  the  driver. 


2. 1.3.1  Scenarios 

The  simulation  scenario  includes  an  accurate  landscape  and  layout  of  the  environment.  The  full  scenario  is 
a  fictive  event  chain  that  is  simulated  and  mediated  in  VBS2™  (see  the  driver  visualization  in  Fig.  7).  The 
group  starts  at  the  CAMP  and  then  follows  a  predefined  pathway  (Fig. 6)  and  reports  position  coordinates 
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the  patrol  chooses  when  going  to  patrol  target  X  (the  governor)  in  the  area,  in  a  town  somewhere  in 
Afghanistan. 

2. 1.3. 2  Training  Sessions 

There  is  no  time  limit  to  the  training  scenario  -  the  purpose  for  the  groups  is  to  train  themselves  out  of 
problems  arising  in  the  simulation  while  physically  being  in  the  home  arena.  Teams  sit  separately  from 
each  other  to  ensure  intercommunication  via  the  simulation.  All  objects  are  representations.  8  training 
scenarios,  or  cases,  A1-A8,  are  shown  on  Figure  8. 


Casel: 

Leave  CAMP 

Case  1-2: 

N  av  igatio  n-Fo  1  low  FI  a  p  sheet 

Case  2: 

Civil  Accident 

Case  2-3: 

Navigation 

Case  3: 

AN  P,  Afghan  National  Police  -Control 

Case  3-4; 

Navigation 

Case  4: 

communication  with  Local  inhabitant  (practice  functions) 

Case  4-5: 

Navigation 

CaseS: 

Roundabout  with  motorcyclist 

Case  5-6: 

Navigation 

Cased: 

ANA,  .Afghan  National  Army -Tank 

Case  6-7: 

Navigation 

Case  7: 

lEO-Bomb 

Case7-S; 

Navigation 

Cased: 

Return  CAM  P 

Figure  8:  The  ISAF  simulation  scenario  cases. 


2.1.2  Second  Field  Study 

The  second  field  study  had  around  25  people  in  training  using  the  same  scenario  as  the  first  field  study. 

The  leader  group  in  the  control  room  (this  time  including  two  technicians  and  two  instructors)  received  a 
Flap  Sheet  (patrol  waypoint  list)  from  the  teams.  The  teams  can  be  followed  via  a  big  screen  in  the 
simulation  room  in  front  of  where  the  leader  group  sits  in  the  control  room.  Communication  regarding 
decisions  was  via  radio. 

A  team  consists  of  4-5  people,  with  one  lead  patrol  of  2-3  officers  and  one  following  vehicle  with  2-3 
officers.  The  lead  vehicle  has  the  PL  for  the  whole  team.  The  second  field  study  had  two  full  teams  in  the 
morning  and  also  two  full  teams  in  the  afternoon.  All  sessions  were  audio  recorded  and  retrospectively  all 
officers  answered  a  questionnaire.  The  analysis  of  the  data  has  first  involved  transcribing  visual  data  and 
sound,  including  communication  between  the  participants. 


3.0  RESULTS 

Detailed  results  presented  here  are  from  field  study  1,  while  the  cognitive  models  developed  later  are 
based  upon  both  field  studies.  This  simulation  took  approximately  1  hour  after  instructions  and  guidelines 
presented  by  the  main  instructor  and  technical  staff.  There  were  two  major  simulation  restarts,  one  back  to 
the  beginning  and  inside  the  camp  area  and  the  second  one  to  a  certain  place  within  the  simulation 
environment. 

The  initial  instructions  by  the  instructor  included:  “The  task  is  to  do  the  vehicle  patrol  in  relation  to  the 
predefined  route  we  have  made,  here  [specified  on  the]  flap  sheet.  If  something  is  happening  along  the 
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road  and  you  both  decide  to  make  a  march  re-plan,  then  you  do  this  concerning  your  decision.  We  do  not 
have  to  think  about  things  as  we  have  to  do  out  in  the  field  and  this  is  what  this  method's  strength  is.  You 
choose  the  way  you  want  to  go,  report  your  re -plan  over  radio,  and  in  relation  to  where  you  are  we  can 
move  the  event  scenarios  we  want  you  to  go  through  irrespectively  what  direction  you  have  chosen.  You 
do  not  have  to  think  about  any  constraints  like  -didn't  he  say  that  we  had  to  follow  the  colored  line?,  no.” 
The  distribution  of  the  instructions  and  its  main  content  are  shown  on  Fig.  9. 


Verbal 
instruct  ions 

-by  inbiructof — 

00:00  01:12  02:24  03:36  04:4£ 


Figure  9:  Categories  of  instructions  before  the  start  of  the  simulated  patrol. 

3.1  Observation 

External  observation  consists  of  seeing  the  external  environment  and  being  able  to  say  when  conditions 
are  normal  and  when  there  are  special  occurrences  of  different  kinds.  Inside  each  vehicle  there  is  a  team 
which  we  can  call  the  internal  team  and  every  person  in  a  team  functions  as  an  observer.  For  one  team 
member,  observation  contains  many  gaze  fixations,  exemplified  by  red  disks  in  the  figures  in  this  section; 
lines  are  saccades,  i.e.  eye  movements  between  fixation  points. 

Individual  fields  of  view  are  combined  to  become  one  that  is  documented  after  the  patrol,  a  common  field 
of  view.  The  PL  role  is  in  charge  of  the  team  and  is  the  main  channel  in  contact  with  the  TOC,  also  s/he  is 
in  charge  of  communication  with  people  in  the  local  community  while  patrolling  as  well  as  being  the 
navigator  in  a  two-man  team. 

The  following  are  examples  of  typical  events  within  the  ISAF  simulation  training  scenarios. 

A1  Leave  Camp;  ISAF  guard  soldiers  control  Vehicle  Patrol,  has  left  Flap  Sheet  copy  in  TOC. 

Opens  the  gate  and  own  staff  drives  out  in  Area  of  Responsibility  (AoR). 

A2  Follow  Flap  Sheet;  Own  staff  passes  (meets)  traffic,  local  inhabitants  and  animals  along  the 

streets. 
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Figure  10:  PL  and  driver  pass  a  woman  in  a  burqa. 


A3  Incident  along  the  road;  Own  staff  meets  a  local  accident.  There  are  people  around  the  local 

vehicle,  and  they  are  searching  for  help. 


Figure  11 :  PL  fixates  above  an  accident  at  distance. 


Figure  12:  Fixations  on  one  of  the  men.  A  man  is  lying  on  the  ground. 


RTO-MP-HFM-202 


17-9 


Decision  Processes  in  Simulation-Based  Training  for  ISAF  Vehicle  Patrols 


otB'f-ix-sjYx;  A'*ii,,r3rjsr 


A4 


Figure  13:  Driver  looks  at  PL.  PL  is  looking  at  the  man  outside. 

Afghan  National  Police  (ANP)  Checkpoint;  Own  staff  is  stopped  at  checkpoint.  The  reason 
is  first  said  to  be  fire  in  some  of  the  quarters  but  the  real  reason  is  due  to  checking  vehicles 
for  weapons  and  if  they  carry  criminals.  The  security  aspect  makes  it  not  possible  for  ISAF 
to  proceed  here  so  re-planning  has  to  be  done  by  the  trainees. 


Figure  14:  A  sign  saying  “SLOW”  and  it  seems  like  a  police  control. 


Figure  15:  PL  fixates  two  of  the  ANP  staff. 


17-10 


RTO-MP-HFM-202 


Decision  Processes  in  Simulation-Based  Training  for  ISAF  Vehicle  Patrols 


Figure  16:  PL  fixates  the  two  ANP  staff  while  communicating 
with  the  third  policeman.  Middle-ground  is  also  attended. 


A5  Re-plan;  Own  staff  chooses  a  new  road,  it  is  reported  to  TOC  so  there  will  be  a  check-up. 


Figure  17:  Measuring  distance  by  gaze. 


Figure  18:  Motorcycle  accident  in  a  roundabout 
and  communication  with  the  motorcyclist. 


A6  Afghan  National  Army  (ANA)  in  wrong  place;  Own  staff  sees  ANA  with  vehicle. 

Questions  arise  about  what  they  are  doing  there? 

A7  Improvised  Explosive  Device  (IED)  observed;  Own  staff  sees  what  is  comprehended  as 

being  an  explosion.  This  happens  close  to  the  camp  that  sends  out  support;  Own  Flap  Sheet 
is  interrupted. 
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Figure  19:  A  car  waiting  for  the  ISAF  vehicle  to  pass. 


Figure  20:  The  same  car  is  overtaking  the  patrol. 


Figure  21 :  Sequence  when  IED  is  thrown  from  the  car  in  front  of  ISAF  vehicle. 


A8  Return  to  CAMP;  Patrol  over. 


Figure  22:  Ground  vehicle  with  PL  and  driver  sees  ISAF  guard  soldiers  again. 

3.1.1  Objects 

Visually  fixated  objects  measured  using  eyetracking  include  people  met  on  the  street,  animals,  stationary 
vehicles,  garbage  piles,  overtaking  cars,  motorcycles  and  bicycles  driving  towards  a  patrol,  stands  with 
fruit  and  vegetables,  own  staff,  etc..  For  all  of  these  simulation  agents/objects  there  are  related  actions  that 
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can  be  actions  of  the  objects  themselves  or  dialog  actions  between  the  team  members.  So  to  understand  the 
dialog,  the  object  is  the  smallest  common  denominator  here  in  this  study. 


Figure  23:  Women,  children,  men,  animals,  and  vehicles  are  passing  by. 


3.1.2  Dialog 

Dialog  is  taken  to  be  primarily  a  description  or  discussion  about  states  where  objects  are  the  basis  for  the 
dialog,  even  if  not  directly  (this  is  a  simplifying  assumption  for  modelling).  Dialog  can  be  divided  into 
three  main  categories: 

•  Dialog  between  the  team  (patrol  leader)  and  the  TOC 

•  Dialog  between  the  team(patrol  leader)  and  local  inhabitants 

•  Internal  dialog  within  the  team,  here  the  driver  and  the  patrol  leader  (e.g.  for  navigation) 

Figure  24  shows  communication  times  with  locals  during  different  kinds  of  meetings,  as  well  as  internal 
communication. 
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Figure  24:  Human  communication  takes  around  12  minutes  in  total  for  the  first  field  study. 
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The  PL  communicated  in  total  08:38  minutes  with  TOC.  The  communication  is  formal  and  changes  in 
form  when  special  events  occur  and  help  of  some  kind  is  needed.  Help  can  consist  of  getting  support  in  a 
special  decision  or  choosing  an  alternative  waypoint. 

The  graph  below,  Fig.  25,  shows  the  distribution  of  duration  in  radio  contacts  with  TOC.  Communications 
have  to  be  short,  formal  and  substantial,  so  everybody  understands  what  is  communicated  if  any 
disturbances  interfere. 
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Figure  25:  Radio  communication  with  TOC  during  a  Vehicle 
Patrol  (time  in  minutes  and  number  of  contact) 


The  total  time  of  communication  between  the  Patrol  Leader  and  Driver  is  24:34  minutes,  see  the 
distribution  between  PL  and  D  in  Fig. 26. 
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Figure  26:  The  number  of  verbal  communications  and  their  length 
in  time  (in  minutes)  between  the  Patrol  Leader  and  the  Driver. 
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As  shown  on  Fig.  24,  the  total  communication  time  with  local  civilians  is  around  7-9  minutes,  while  the 
remaining  time  outside  the  team  is  communication  with  local  police.  We  have  a  threat  scenario  following 
the  motor  cycle  accident  that  is  very  short,  just  16  seconds,  where  the  driver  tells  the  PL  to  immediately 
return  to  the  vehicle  so  the  driver  can  reverse.  Just  after  this,  two  men  throw  stones  at  the  vehicle  and  TOC 
is  contacted  directly  afterwards.  They  are  instructed  by  TOC  to  head  back  to  the  camp. 

3.2  Objects  and  Dialog 

The  communication  carried  out  in  this  context  seems  to  build  on  the  stimulus  of  a  gazed  object  with  dialog 
following  gaze  behavior.  The  dialog  concerns  situation  estimation,  deliberation  over  what  to  do  (actions), 
and  how  to  approach  inhabitants  on  the  spot.  Not  all  objects  have  any  importance  but  still  many  gazed 
objects  have  to  be  deliberated  in  case  they  may  hide  information,  can  be  of  harm,  etc. 
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Figure  27:  An  example  of  how  objects  and  dialog 
(including  meta-controls!)  are  related  in  this  context. 
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Figure  28:  This  is  the  situation  with  the  moped  and  the  man  claiming  $500.  In  the 
background  quite  many  people  appear  to  be  lining  up  to  support  the  man. 


3.3  Cognitive  Models 

The  previous  section  has  provided  examples  of  the  data  obtained  primarily  in  field  study  1.  This  data, 
together  with  audio  recordings  and  answers  from  field  study  2,  is  the  basis  for  cognitive  modelling 
foundations  summarized  in  this  section.  The  primary  aim  here  is  not  to  present  detailed  cognitive  models. 
Rather,  the  outcome  of  this  pilot  study  is  the  identification  of  the  abstract  form  of  models  (more 
accurately,  the  cognitive  modelling  metamodels)  required  for  comprehensive  modelling  of  decision 
processes  within  this  domain. 


Figure  29:  BDI,  a  cognitive  reasoning  model,  and  its  modification  over  several  steps  from 
individual  and  organizational  points  of  view,  education  through  operational  missions  in  teams, 
to  retrospective  performance  evaluation.  The  last  step  is  gathered  competency  that  can  feed 
back  into  the  high  level  BDI  model  of  the  system  and  organization  again. 


The  cognitive  modelling  methodology  and  software  called  Beliefs,  Desires  and  Intentions  (BDI, 
Wooldridge,  2000)  functions  as  the  starting  point  for  cognitive  modeling  in  this  pilot  study.  However,  BDI 
has  not  been  found  to  be  sufficient  for  modeling  all  aspects  of  decision  processes  in  the  military 
application  domain.  Hence  the  process  models  shown  on  Fig.  29  begin  with  BDI,  but  then  present  a  series 
of  variants  resulting  from  working  with  BDI  and  interpreting  it  from  points  of  view  of:  the  military 
organization,  education,  instructors,  simulation-based  training,  operational  scenarios  in  the  field,  teams, 
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and  debriefing  situations.  The  resulting  BDI-derived  cognitive  modeling  methods  developed  in  this  study 
are  (Fig.  29): 

•  The  original  BDI  and  its  framing 

•  The  AksGA  model  for  training  in  military  education 

•  OGA,  a  simulation-mediated  perspective 

•  OPA/OPdA  and  OPdAt  using  VBS2  and  eyetracker/adaptation  emphasizing  object  perception 
and  taking  advantage  of  rapid  eye  gaze 

•  IDCto  where  shared  imperfect  experience  and  debriefing  are  major  cornerstones 

These  models  are  interrelated  and  are  understood  to  operate  in  parallel  where  many  levels  of  operational 
and  training  environments  and  systems  are  considered.  Note  in  particular  that  these  are  not  implementation 
models;  the  question  of  the  best  architectures  and/or  languages  for  the  implementation  of  the  models  is  not 
within  the  scope  of  this  paper.  Rather,  the  models  are  abstractions  of  distinct  decision  processes  within  the 
application  domain,  derived  from  the  field  studies.  Application  of  all  of  these  models  (which  has  not  yet 
been  done,  and  therefore  remains  to  be  validated)  is  a  suggestion  for  how  to  contribute  to  system 
development  where  learning,  sociability,  time  issues  and  technology  can  provide  tools  to  meet 
requirements  for  decreasing  gaps  in  training,  and/or  increasing  safety  and  security  in  open  and  closed 
worlds,  or  a  mix  of  both. 

3.3.1  BDI 

In  this  study  it  has  been  found  that  the  original  BDI  model  has  been  found  to  stand  for  the  organizational 
framework  within  which  simulation-based  training  is  conducted  and  how  the  framework  and  rules  are 
formed  over  many  years  such  that  the  organization  stands  for  certain  values  and  rules,  that  all  individual 
activities  within  the  organization  are  supposed  to  conform  to.  So  we  can  say  it  is  an  intrapersonal2  as  well 
as  an  organizational 3  model. 

The  word  desire  here  is  about  motivation4  of  goal-oriented  behavior,  a  thought  that  leads  to  action.  The 
thought  can  vary  depending  on  what  kind  of  situation  we  may  meet.  Desires  do  not  really  have  a 
functional  place  when  it  comes  to  fast  action  in  operational  tasks  in  a  changing  environment,  since  fast 
action  responses  are  reflexive,  prebuilt  during  training;  the  routines  and  the  intensions  of  a  certain 
framework  are  worked  out  beforehand  and  discussed  so  they  should  not  need  elaboration  while 
performing  varied  tasks  in  the  field  that  are  not  necessarily  routine,  but  at  least  anticipated. 

Beliefs,  desires  and  intentions  are  therefore  perceived  here  as  a  model  that  lies  as  a  background  for  an 
organization,  where  frameworks  are  developed  over  long  periods  of  time  and  also  where  documentation 
and  negotiation  are  important  processes  for  forming  a  stable  framework.  The  pathos  of  the  organization 
and  its  contribution  to  a  peace  process  is  of  high  importance. 

This  belief-desire-intention  framework  is  what  an  individual  is  "accepting"  when  deciding  whether  to  join 
an  organization  or  not.  The  values  and  the  ideology  in  the  case  of  the  UN  charter  and  peacekeeping 
missions  stand  for  hope,  good  and  human  peace  (United  Nations,  2008).  On  the  intrapersonal  level,  beliefs 
are  elaborated  and,  depending  on  private  status  concerning  being  single,  divorced,  married,  having  any 
children,  kind  of  religion  or  not,  it  may  be  easier  or  harder  to  take  a  decision  to  join  and  commit  to  the 
organization.  Desires  probably  play  a  major  part  in  this  and  can  include:  attending  deployment  again 


2 

http://www.mypersonality.info/multiple-intelligences/intrapersonal/,  accessed  2010-05-05 
http://www.cognitivebehavior.com/management/concepts/organizationalmodels  table.html,  accessed  2010-05-05 

4  http://en.wikipedia.org/wiki/Motivation,  accessed  2010-05-05 
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because  it  was  a  former  good  experience,  to  earn  money,  to  make  a  contribution  to  a  messy  world,  to  go 
away  from  a  family  situation,  to  join  because  other  colleagues  have  decided  to  do  so,  career  advancement, 
etc.  The  intension  then  is  actually  to  go  on  a  mission  (or  not). 

Once  a  person  has  joined  a  military  organization,  from  the  organizational  and  intrapersonal  layer  they  will 
be  trained  and  prepared  by  the  organization.  Instructors  and  staff  will  train  their  future  staff  before  they 
leave  their  home  country.  Officers  joining  have  former  schooling  in  the  military  and  an  awareness  of  the 
organization,  but  maybe  have  not  been  operating  in  an  international  mission  before,  or  with  ISAF  forces. 
Hence  the  BDI  organizational  layer  frames  the  whole  organization  and  mission,  and  mission  training 
includes  training  in  the  values  of  this  layer. 

The  BDI  model  has  constraints  if  we  use  it  for  operational  purposes  in  fast  paced  decisions.  BDI  is 
perfectly  fine  from  an  organizational  and  an  intrapersonal  point  of  view,  but  when  it  comes  to  operational 
tasks  in  situations  of  observation  with  ROE  as  a  framework,  we  have  to  think  of  a  model  of  another  kind. 

3.3.2  AksGA 

A  stands  for  Ability,  ks  for  Knowledge  and  Skill(s),  G  stands  for  Goal  and  A  for  Action.  The  abilities 
officers  are  to  practice  have  to  function  as  background  knowledge,  a  backbone  plan,  which  will  be  used  in 
most  activities  carried  out  in  an  international  operational  mission.  The  trained  AksGA  plans  are  those  the 
officer  applies  to  know  how  to  act  in  certain  situations. 

AksGA  includes  the  ability  model  used  in  the  military  education  in  Sweden  (Forsvarsmakten,  1988,  2006). 
The  ability  model  is  used  for  educating  military  staff  both  for  national  and  international  missions,  with  the 
aim  of  making  education  effective  at  least  in  part  by  having  people  use  the  same  concepts  and  having  a 
collective  language  for  goal  formulations  in  qualitative  terms.  The  ability  model  is  used  to  shape  a 
comprehension  of  simulation-based  training  sessions  for  upcoming  missions. 

Bratman’s  BDI  model  (Bratman,  1999/87)  is  captured  in  IntUtbE’s  principle  of  HOW  TO  DO  which 
could  be  said  to  be  based  in  Belief  plus  Desire  which  then  gives  the  Intention  to  perform  or  to  do 
something.  However,  the  result  of  the  training  process  is  to  combine  APPLICATION  (ability)  as 
KNOWLEGDE  and  SKILL  with  educational  goals  so  the  trainees  know  how  to  act  in  different  situations, 
e.g.  how  to  stop  blood  pumping  out  from  an  artery  or  how  to  shoot,  how  to  use  a  vehicle,  a  weapon,  a 
system,  etc..  The  educational  model  means  that  a  trained  person  has  done  an  action  before  and  can  do  it 
independently  and  also  that  time  pressure  can  come  into  the  picture  without  being  a  problem.  Hence  these 
learning  outcomes  are  more  suitably  modelled  using  AksGA  as  an  extension  of  the  BDI  model. 

On  a  group  level,  AksGA  models  collective  action  of  people  with  different  experiences  after  formal 
education  about  a  certain  area  or  context.  From  an  educational  point  of  view,  the  aim  is  to  have  everybody 
up  to  a  common  collaborative  lowest  level  in/for  a  context.  So  with  AksGA,  we  move  from  the 
organizational  and  intrapersonal  BDI  to  personal  and  group  training  outcomes  concerning  action-oriented 
abilities  and  skills.  The  AksGA  model  in  its  context  helps  to  clarify  performance  qualities  and  quality 
levels  for  an  individual  as  well  as  for  a  team.  The  description  and  use  of  the  ability  model 66 . .  .is  especially 
useful  in  goal  formulations  as  well  as  planning,  performance,  analysis  and  evaluation  of  education” 
(Forsvarsmakten,  1988).  AksGA  is  an  abstraction  of  the  structure  and  functions  of  the  ability  model. 

3.3.3  OGA 

The  Object,  Goal  (in  context)  and  Action  (OGA)  stage  operates  in  a  training  or  adjustment  phase  where 
there  are  no  real  world  consequences.  The  original  BDI  model  has  its  percept  layer  outside  the  model, 
while  in  OGA  this  is  lifted  into  the  model  as  “O”,  the  object  layer.  This  layer  replaces  Belief  with  tangible 
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object  perception.  The  object  can  of  course  be  any  object  including  vehicles  or  weapons  one  is  educated  to 
handle.  “G  (in  context)”  adds  the  dimension  of  what  kind  of  world  and  regional  situation  holds,  so  the 
team  knows  how  to  behave  and  in  relation  to  the  general  state  they  can  elaborate  what  actions  they  are 
permitted  to  take.  Goal  development  involves  typically  long  term  interaction  and  a  bridge  from  ability 
training  and  long  term  negotiations  into  every  day  operations,  a  state  of  peacekeeping  operations  where 
the  OGA  can  be  a  prolongation  of  the  BDI  and  where  also  both  models  can  work  in  parallel. 

We  have  now  gone  from  BDI  to  AksGA  to  OGA  (Object,  Goal  and  Action),  where  Belief  is  replaced  with 
Object  as  the  source  of  information,  Desire  is  replaced  with  Goals  and  Intention  is  replaced  with  Action. 
This  means  that  certain  tasks  in  relation  to  The  state  of  the  world"  change.  In  simulation-based  training 
this  model  is  more  accurate  when  the  updates  of  the  engine  with  its  rendering  shape  another,  faster  pace 
for  exercising  decisions  than  for  the  BDI  or  AksGA  models.  The  officer  observes  the  world,  in  this  case 
scenarios  in  VBS2,  and  the  simulation  world  contains  objects  of  various  kinds.  An  OGA  model  provides 
the  mapping  from  these  observations  to  chosen  (and  potential)  actions  according  to  relevant  goals. 

The  OGA  model  brings  the  3D  percept  into  the  model  instead  of  keeping  it  external.  One  argument  for 
building  the  percept  layer  into  the  model  is  to  save  time  for  people  working  in  time  critical  situations  and 
that  can  mean  to  save  lives.  To  earn  seconds  or  fractions  of  seconds  can  be  critically  important  for 
individuals,  teams  and  organizations.  This  kind  of  model  can  of  course  log  what  is  gazed  in  a  fast  paced 
environment,  but  if  you  can  log  then  you  can  also  steer  by  gaze  if  you  are  occupied,  etc..  This  could  be 
used,  for  example,  as  a  help  device  in  negotiation,  life  critical  situations,  or  if  your  hands  are  occupied  and 
you  have  to  act  fast  with  maybe  no  help,  etc..  This  functionality  can  be  used  both  for  training  and  for 
operational  purposes  where  technology  for  augmentation  can  support  and  enhance  human  performance. 
But  of  course,  this  is  far  too  much  detail  for  higher  level  concerns  of  the  organization,  which  here,  it  is 
proposed,  are  better  modelled  using  BDI,  while  AksGA  is  more  concerned  with  learning  social 
collaboration  and  providing  a  map  of  different  kinds  of  abilities,  knowledge  and  skills  as  training  targets, 
without  necessarily  going  into  the  details. 

The  question  arises  of  how  committed  an  agent  should  be  when  the  agent  has  selected  an  action.  This  may 
involve  two  questions:  1)  how  committed,  and  2)  for  how  long?  There  are  different  kinds  of  commitments 
(Wooldridge,  2000):  i)  blind  commitment/fanatical  commitment,  ii)  single-minded  commitment,  and  iii) 
open-minded  commitment.  The  OGA  level  allows  for  some  extended  deliberation  in  order  to  resolve  these 
kinds  of  issues  and  obtain  a  decision. 

3.3.4  OPA  and  OPdA 

Some  actions  have  purposes  that  may  not  drive  deliberative  action  selection  by  individuals  or  teams,  for 
example,  routines,  such  as  reporting  reaching  waypoints  to  TOC.  Routines  are  trained  for  and  performed 
often  enough  that  they  are  performed  automatically  and  with  little  deliberation,  and  without  being  driven 
by  detailed  conscious  goals.  They  relate  to  specific  objects,  have  purposes,  and  consist  of  highly  familiar 
actions.  This  can  be  modelled  using  Objects  (O),  Purposes  (P)  and  Action  (A).  This  is  referred  to  as  an 
OPA  model. 

It  is  also  necessary  that  at  least  some  training  aims  for  direct  and  rapid  action  selection  under 
circumstances  that  are  not  routine  (although  they  may  be  heavily  trained  for)  without  any  extended 
deliberation5.  Hence,  while  an  action  in  OGA  can  be  regarded  as  a  deliberatively  chosen  behavior  in 
response  to  certain  stimuli,  interaction  (as  in  AksGA)  is  more  of  a  dialog,  and  direct  action ,  dA,  is  more  or 
less  reacting  with  highly  automated  responses  and  little  or  no  deliberation.  To  accommodate  this  we 
introduce  the  Object,  Purpose,  (direct)Action,  or  OPdA,  model.  (Actions  in  OPA  have  less  urgent  time 
requirements  than  in  OPdA.)  OPdA,  stands  for  a  level  of  direct  action  that  may  be  modified  by  a 
deliberative  context. 


5  This  means  a  spine  decision,  or  reflex  reaction,  achieved  by  training. 
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In  the  field  studies,  statements  were  observed  that  were  about  scenarios  testing  cognitive  decision-making 
and  judgment  rather  than  skills,  which  should  eventually  lead  to  the  capacity  for  skill-based  direct  action 
when  there  is  no  room  for  cognitive  decision-making  in  the  actual  moment,  or  direct  action  responses  may 
be  trained  in  a  way  that  does  not  rely  upon  higher  level  cognitive  processing  other  than  an  understanding 
of  the  context.  In  direct  action,  cognitive  procedures  and  allocations  have  been  trained  and  built  up  over 
many  hours  and  possibly  years.  Reaction  can  mean  survival  and  is  a  fast  response.  This  can  also  be  to 
rescue  others,  and  all  roles  within  military  mission  training  (including  for  health  and  emergency  staff)  are 
given  weapon  education  as  a  common  ground  for  self-defence  situations.  The  object  and  its  value  are  what 
an  agent  sees  (O),  the  information  about  the  world,  and  the  purpose  (P)  of  action  (dA)  is  to  meet  that 
object  and  its  value  in  a  way  that  achieves  higher  level  goals,  but  without  deliberative  action  selection. 

From  a  BDI  organizational  perspective  the  information  about  the  world  and  its  framework  is  quite  stable, 
while  a  decision  under  pressure  (at  the  OPdA  level)  brings  us  to  the  question  of  how  to  quickly  process  the 
information  at  hand  to  arrive  at  an  action  while  staying  within  the  framework  of  rules  and  negotiations. 
The  VBS2™  simulation-based  training  data,  gaze  recording  and  the  field  studies  at  SWEDINT  provide  a 
basis  for  detailed  descriptions  of  the  components  required  to  develop  more  detailed  OPA/  OPdA  models. 
These  are  described  in  Sennersten  (2010). 

3.3.5  OPdAt 

From  OPA  and  OPdA  we  have  a  comprehension  of  how  Object(s),  Purposes  and  Actions  function  and  are 
thought  of.  These  models  have  been  derived  from  observations  of  individual  roles,  but  we  must  also 
consider  how  a  team  may  function.  This  leads  to  a  team  extension  to  the  OPdA  model,  leading  to  OPdAt, 
where  t  stands  for  Team.  There  are  different  kinds  of  teams  and  here  the  term  team  will  be  used  with  the 
meaning  of  “A  group  organized  to  work  together  and  a  group  on  the  same  side  (as  in  a  game)”6. 

OPdAt  captures  training  for  the  ability  to  constitute  a  team  in  relation  to  operational  tasks.  The  team 
participants  are  required  to  understand  individual  roles  and  their  prerequisites,  leading  to  a  cognitive  team 
model7 .  As  a  result  of  this  training,  “job  performance  (task  performance)  is  related  to  cognitive  ability  and 
not  to  personality”  (LePine  &  Dyne,  2001)8.  OPdAt  allows  modelling  of  individual  and  possible  collective 
“observation”  tasks  that  can  result  in  a  common  cognitive  model  for  specific  kinds  of  decisions.  Direct 
action  here  means  to  act  and  know  what  to  do  in  a  coordinated  way  among  team  members  without  long 
thought  processes  or  individual  or  collective  deliberation.  Coordination  among  team  members  may 
involve  terse  verbal  or  nonverbal  signs.  The  major  difference  among  members  of  a  patrol  team  is  the  role 
allocation  and  the  distribution  of  observation  areas  between  roles,  with  one  person  being  in  charge  overall 
for  decisions. 

3.3.6  IDCxo 

Information,  Debrief  and  Competency  within  Team  and  Organization  (IDCto),  refers  to  collective 
cognitive  processing  where  after  training  and  debriefing  the  team  has  developed  an  imperfect  competency 
together  that  can  be  of  further  use  for  later  team  collaboration  in  improving  understanding.  The 
competency  is  an  interpersonal  shared  mental  model  based  upon  experience  of  situations.  It  is  important 
here  to  understand  the  difference  between  imperfect  post-experiential  information  and  information  needed 
in  ongoing  task  performance  in  the  present.  Debriefing  is  about  role-  and  team-shaped  postscripts  in 

6  http ://w w w.thefreedictionary. com/team,  accessed  2010-05-05 

7 

http://www.google.com/books ?hl=sv&lr=&id=U  8GVml  6IMC&oi=fnd&pg=PA221&dq=cognitiye+team+model&ots=VqY 

PanRJ86&sig=iTHogFsiChPolP0vYk-PRlClhgE#v=onepage&q=cognitive%20team%20model&f=false,  accessed  2010-05- 
05 

g 

http://www.personalityresearch.org/papers/neubert.html,  accessed  2010-05-05 
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relation  to  former  initial  prescripts.  The  pre-  and  postscripts  form  learning/comprehension  outcomes.  The 
simulation-based  training  scenarios  carried  out  provide  a  basis  for  case  studies  in  debriefing  and  also 
provide  a  ground  for  in-simulation  team  dialog. 


4.0  POTENTIAL  APPLICATIONS  OF  COGNITIVE  MODELS 

4.1  Training  Evaluation 

The  cognitive  (meta)  models  presented  above  identify  different  levels  and  forms  of  cognitive  decision 
processes  involved  in  military  simulation-based  training.  Detailed  models  at  these  levels  can  provide  a 
basis  for:  i)  analysing  and  understanding  cognitive  processes  both  within  training  environments  and  real 
operational  theatres,  including  evaluation  of  the  effectiveness  of  different  operational  decision  models,  ii) 
expressing  target  cognitive  processes  for  training,  and  iii)  evaluating  the  effectiveness  of  training  systems 
by  comparison  of  target  models  with  models  achieved  via  training  systems.  These  three  uses  represent  a 
development  sequence,  of  first  identifying  cognitive  decision  model  variants  and  possibilities,  then 
prioritizing  and  selecting  from  these  to  specify  detailed  cognitive  training  targets,  and  then  using  these 
targets  for  evaluating  and  improving  training  processes.  Of  course,  fully  developing  comprehensive 
models  at  all  levels  for  a  given  operational  environment  would  be  an  enormous  and  perhaps  impossible 
task,  so  actual  use  of  the  models  will  require  more  selective  application. 

4.2  Training  and  Operational  Enhancement 

A  team  in  a  simulation-based  training  environment  interacts  with  visual  content  produced  by  simulation 
and  rendering  engines  via  devices  and  sensors.  Having  an  explicit  representation  of  target  cognitive 
models  for  training  leads  to  the  possibility  of  implementing  those  models  within  the  training  system.  This 
could  support  real-time  mapping  and  analysis  of  trainee  performance  against  those  target  models, 
providing  feedback  for  instructors  to  adapt  the  simulation  to  provide  more  training  in  less  developed  areas 
of  trainee  cognitive  ability.  After  the  simulation,  mapping  performance  against  a  target  model  can  provide 
detailed  information  on  trainee  performance  for  feedback,  discussion  and  ongoing  training.  Such  a  model 
could  also  be  used  to  automatically  adapt  the  simulation  in  real  time,  for  example,  by  providing  more 
examples  for  areas  where  further  training  is  needed  and  progressively  increasing  task  difficulty  as  trainee 
ability  increases  (a  common  principle  of  computer  game  design). 

More  significant  enhancements  can  be  expected  if  various  psychophysiological  inputs  can  also  be  included 
in  simulation  interaction.  In  principle  these  could  include  galvanic  skin  response  (GSR)  measuring 
arousal,  electromyography  (EMG)  measuring  emotional  valence,  heart  rate/pulse,  and 
electroencephalography  (EEG)  measuring  approach/avoidance,  attention,  etc.  (see,  for  example, 
Andreassi,  2006).  Such  inputs  can  provide  data  about  emotional  states  and  their  role  in  cognitive  decision 
processes,  including  the  effectiveness  of  approaches  to  minimizing  the  influence  of  emotions  on 
instrumental  task  performance  and  decisions  that  need  to  be  driven  by  operational  needs. 

A  training  system  taking  psychophysiological  measures  into  account  at  the  same  time  as  pre-plans,  actual 
plans  and  carried  out  plans  can  be  tracked,  can  make  the  team  members  more  aware  of  themselves  and 
each  other,  and  also  support  decisions  in  situations  where  the  world  update  changes  rapidly.  A  possible 
interface  proposed  here  for  providing  such  feedback  is  a  Team  radar"  (see  Fig. 30),  that  can  function  for 
training  support  and  cross  correlate  important  states  of  action.  A  team  radar  implemented,  for  example,  in 
VBS2™,  (see  Fig.  31),  would  graphically  represent  initial  personal  data  like  weight,  pulse  (rest-pulse), 
age,  mood  and  emotion  (derived  from  psychophysiology  data),  role,  trust  thresholds,  skill(s)  etc..  The 
outcome  of  this  feedback  can  be  increased  awareness  of  the  status  of  the  group,  and  individual  soldiers 
and  officers  can  be  empowered  in  their  roles  when  taking  hard  decisions  when  they  arise,  by  mapping 
action  decisions  to  optimal  performance  models  at  the  different  cognitive  levels  described  above. 
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Figure  30:  Proposed  Team  Radar'. 


Psychophysiological  interaction  does  not  require  any  efforts  by  the  trainee  because  sensors  gather 
physiological  data  without  any  conscious  input  from  the  user.  Software  tools  can  categorize  and  cross¬ 
correlate  data  so  it  can  become  usefully  displayed  and/or  used  in  real-time,  as  well  as  logged  and  analysed 
retrospectively. 

Eye  gaze  interaction,  physiological  tracking  and  team-based  cognitive  modelling  and  execution  software 
such  as  JACKTeam™  can  create  the  basis  for  comprehension  of  interaction  patterns  that  help  in  debriefing 
and  how  to  build  safer  and  more  effective  operational  systems  for  people  in  missions.  The  AOS  company 
has  implemented  CoJACK™  within  VBS2™.  CoJACK  can  model  human  cognition,  including  the  effects 
of  moderators  (e.g.  fatigue,  fear)  on  human  performance,  with  JACKTeam™  supporting  autonomous 
teams.  Each  team  exists  as  an  entity  with  separate  beliefs  from  those  of  its  constituent  agents.  The 
software  supports  different  teaming  algorithms  and  allows  the  representation  of  social  relationships  and 
co-ordination  between  team  members.  Team-oriented  programming  is  presented  as  an  intuitive  paradigm 
for  engineering  group  action  in  multi-agent  systems.  The  software  can  then  specify  what  a  team  is  capable 
of  doing  and  which  components  are  needed  to  form  a  particular  type  of  team,  and  then  provide 
coordinated  behavior  among  the  team  members  and  team  knowledge. 
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Figure  31 :  Instruments  interconnected  to  the  VBS2  engine,  with  CoJACK™, 
JACKTeam™,  an  eyetracker,  GSR,  pulse,  and  speech  inputs. 


In  the  simulation-based  training  environment  proposed  in  Fig.  31,  before  starting,  and  depending  upon  the 
physical  constraints  of  the  operational  or  training  environment,  all  team  members  place  themselves  in 
front  of  an  eyetracker  which  then  is  interconnected  with  the  VBS2™  engine,  CoJACK™  and 
JACKTeam™,  and  are  also  provided  with  a  pulse  watch,  GSR  electrodes,  and  a  microphone  for  speech 
recognition.  During  activity,  gaze,  objects,  and  audio  will  stand  for  transformations  through  or  depending 
on  every  participant’s  physiological  reactions.  How  actions  conform  or  not  to  prior  training  can  be 
assessed  and  fed  back,  both  to  the  individual  and  the  team  so  that  team  self -regulation  becomes  an  explicit 
cooperative  goal  for  the  group. 


5.0  CONCLUSION 

We  have  presented  a  pilot  study  of  simulation-based  training  developed  by  SWEDINT  for  IntUtbE  for 
ISAF  troops  in  preparation  for  ground  vehicle -based  patrols  in  Afghanistan.  The  pilot  study  involved  two 
field  studies  and  resulted  in  a  metamodel  providing  a  representation  paradigm  for  the  more  detailed  kinds 
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of  cognitive  models  required  to  represent  decision  processes  and  their  context  both  for  training  and  in  the 
target  theatre  of  operations.  This  metamodel  provides  a  more  specific  foundation  for  ongoing  detailed 
modelling,  both  for  evaluation  of  simulation  effectiveness  and  potentially  for  enhanced  functions  within 
both  simulation-based  training  and  actual  operations.  It  has  further  been  proposed  in  this  paper  that  greater 
benefits  may  be  realized,  both  in  training  and  in  operations,  by  the  introduction  of  real-time 
psychophysiological  monitoring  and  feedback  for  troops,  instructors  and  commanders,  integrated  with 
cognitive  models.  Extensions  of  this  concept  could  involve  augmented  reality  systems  for  direct  feedback 
of  group  state,  e.g.  using  a  team  radar.  The  deployment  of  both  centralised  and  distributed  cognition 
models  is  fundamental  for  enhanced  operations,  and  for  the  successful  integration  of  both  manned  and 
unmanned  combat  agents.  Such  a  vision  is  fully  compatible  with,  and  in  fact  heavily  dependent  upon,  the 
heavily  internetworked  architecture  of  modern  western  combat  systems;  the  aim  of  this  paper  has  been  to 
use  this  architecture  as  a  foundation  for  enhanced  training  effectiveness  by  providing  and  representing 
new  dimensions  of  situational  awareness  in  terms  of  cognitive  state,  and  effective  synchronization  of  the 
cognitive  foundations  of  operations  at  multiple  levels  of  organizational,  temporal  and  spatial  resolution. 
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